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(54) TRAVEUNG OBJECT SENSING DEVICE 
(57)Abstract: 

PROBLEM TO BE SOLVED: To sense a traveling object 
such as a vehicle stably independently of a time change 
or a weather condition change and to reduce the 
opportunity of misdetection of the object due to noise. 
SOLUTION: Input images acquired sequentially by an 
image sensor 1 in timely different timings are compared 
respectively to obtain a difference. When significant data 
of a prescribed quantity or over are in existence in 
image data obtained resulting from applying AND 
operations to the difference, intrusion of a mobile object 
into a supervised area is discriminated. Upon the 
detection of the traveling object, an intruded vehicle 
detection section 4 takes a difference from a 
background image to generate its template. The 
background image is obtained by applying weight sum 
averaging processing to the image in the supervised 
area. 
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CLAIMS 



[Claim(s)] 

[Claim 1] The image which the image sensor which acquires the image of a fixed 
monitor field, and this image sensor acquired is analyzed. It has the penetration 
detecting element which carries out detection processing of the mobile which 
advances into said monitor field. This penetration detecting element the input image 
acquired in order to the timing from which said image sensor differs in time — as 
the 1st image, 2nd image, and 3rd image — accepting — the difference of the 1st 
image and the 2nd image — with a value the difference of the 2nd image and the 3rd 
image — the mobile detection equipment characterized by calculating a value, and 
judging that the mobile advanced into said monitor field if the significant data more 
than a constant rate exist in the image data obtained by the AND operation of said 
error-due-to-curvature-and-refraction part value. 

[Claim 2] Mobile detection equipment characterized by having the background 
creation section which updates the background image of a monitor field saved for 
comparison processing each time when a penetration detecting element judges that 
the mobile is not advancing into a monitor field as a result of an AND operation in 
mobile detection equipment according to claim 1. . 

[Claim 3] the difference of the penetration detecting element which carries out 
detection processing of the mobile which analyzes the image which the image sensor 
which acquires the image of a fixed monitor field, and this image sensor acquired, 
and advances into said monitor field, and a background image when this penetration 
detecting element detects a mobile, in case a mobile is not and an input image ~ 
the mobile detection equipment characterize by to have take out the image of the 
lower part of a mobile from a value, and to have the template creation section which 
creates a template. 

[Claim 4] The mobile detection equipment characterized by to have the template 
creation section which picks out the image of the specific part of a mobile from an 
input image, and creates a template when the penetration detecting element which 
carries out detection processing of the mobile which analyzes the image which the 
image sensor which acquires the image of a fixed monitor field, and this image 
sensor acquired, and advances into said monitor field, and this penetration detecting 
element detect a mobile. 

[Claim 5] It is mobile detection equipment characterized by creating the label image 




which gave the label value for distinguishing each nnobile, respectively to two or 
more mobiles by which the template creation section was detected in the image in 
mobile detection equipment according to claim 3, starting the part of a label field 
with the same label value included in this label image which includes a lower limit at 
least, and creating a template. 

[Claim 6] It is mobile detection equipment which makes an invalid the label value of 
the label field for the mobile concerned, and is characterized by distinguishing from 
the label field for the mobile which is not yet creating the template after the 
template creation section creates a template In mobile detection equipment 
according to claim 5. 

[Claim 7] It is mobile detection equipment characterized by for the template 
creation section detecting the image of the lower limit of a mobile from an input 
image in mobile detection equipment according to claim 3. and creating the template 
of the mobile when the distance in the image from the lower limit of an input image 
to the lower limit of a mobile is beyond a threshold. 

[Claim 8] It is mobile detection equipment which the template creation section 
detects the image of the lower limit of a mobile from the penetration car detection 
field in an input image in mobile detection equipment according to claim 3, and is 
characterized by cutting down the image of the lower part of the mobile from said 
input image, and creating a template when the distance in the image from the lower 
limit of a penetration car detection field to the lower limit of a mobile is beyond a 
threshold. 

[Claim 9] Mobile detection equipment with which the color or luminance distribution 
of the image seen horizontally is characterized by judging the part which fulfills 
predetermined conditions to be the lower part of a mobile in mobile detection 
equipment according to claim 3 when the image of a mobile is detected from an 
input image. 

[Claim 10] The image sensor which acquires the image of a fixed monitor field with a 
predetermined frame period, the image of two or more monitor fields acquired to the 
timing from which this image sensor differs in time — that difference — it analyzing 
in quest of a value, and, when the penetration detecting element which carries out 
detection processing of the mobile which advances into said monitor field, and this 
penetration detecting element detect a mobile In the image of two or more monitor 
fields acquired to the timing to which said image sensor differs from the template 
creation section which picks out the image of the lower part of a mobile from an 
input image, and creates a template in time Mobile detection equipment 




characterized by having the rate test section which asks for the passing speed of a 
mobile from the trace section which carries out matching processing using said 
template, and detects the movement magnitude of a mobile, the movement 
magnitude of the mobile which said trace section detected, and said frame period. 
[Claim 11] It is mobile detection equipment characterized by receiving an input 
image with such a short frame period that the trace section having many templates 
in mobile detection equipment according to claim 10, and receiving an input image 
with such a long frame period that there being few templates. 
[Claim 1 2J It is mobile detection equipment characterized by receiving an input 
image with such a short frame period that the passing speed of the mobile by which 
the trace section was detected in the input image in mobile detection equipment 
according to claim 10 being quick, and receiving an input image with such a long 
frame period that passing speed being slow. 

[Claim 1 3] About the image of two or more monitor fields acquired to the timing 
from which the image sensor which acquires the image of a fixed monitor field, and 
this image sensor differ in time that difference it analyzing in quest of a value, 
and, when the penetration detecting element which carries out detection processings 
of the mobile which advances into said monitor field, and this penetration detecting 
element detect a mobile In the image of two or more monitor fields acquired to the 
timing to which said image sensor differs from the template creation section which 
picks out the image of the lower part of a mobile from an input image, and creates a 
template in time Mobile detection equipment characterized by having the rate test 
section which asks for the passing speed of a mobile from the trace section which 
carries out matching processing using said template, and detects the movement 
magnitude of a mobile, the movement magnitude of the mobile which said trace 
section detected, and said frame period. Mobile detection equipment characterized 
by having the background creation section which obtains the background image a 
mobile is not in a monitor image by weighted average processing of the image of two 
or more monitor fields acquired to the timing from which said image sensor differs in 
time. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 




[Field of the Invention] This invention relates to the mobile detection equipment for 
detecting a motion of mobiles, such as a car, which moves in the fixed field made 
applicable to a monitor. 
[0002] 

[Description of the Prior Art] Road traffic capacity is made to expand, a traffic 
accident is controlled, and research of a next-generation road transportation system 
is briskly recommended for the purpose of attaining energy saving. The technique 
which carries out automatic detection of the motion of a car path on the street etc. 
correctly is one of the techniques indispensable to this next-generation road 
transportation system. The image sensor for photoing the fixed field made applicable 
to a monitor is used for the car detection equipment used for such a purpose. By 
this image sensor, a wide range field is supervised and the image of a mobile is 
acquired. This image is analyzed and the motion of a car which passes through that 
section is detected. Such a technique is introduced to "Takahashi, Kitamura, the 
Kobayashi work, and 'research of traffic-flow monitor approach by image 
processing" Shingaku Giho Vol.97 No.40 PRMU 97-7 (1997)." 
[0003] 

[Problem(s) to be Solved by the Invention] By the way, there were the following 
technical problems which should be solved in the above Prior arts. When analyzing 
by acquiring the image of mobiles, such as a car which moves in a monitor field top, 
with the image of the background region, it is necessary to distinguish two or more 
mobiles, always updating the background image which changes according to change 
of time amount or the weather. Moreover, although the template which cut down 
only the image of a mobile part must be acquired in order to detect the number and 
passing speed of a mobile which move in the inside of a monitor field, the template 
which can be clearly distinguished from other mobiles must be started out of the 
image of a monitor field. Development of a more concrete practical approach is 
desired about such a point in development of a next-generation road transportation 
system. 
[0004] 

[Means for Solving the Problem] This invention adopts the next configuration in 
order to solve the above point. 

<Configuration 1> The image which the image sensor which acquires the image of a 
fixed monitor field, and this image sensor acquired is analyzed. It has the penetration 
detecting element which carries out detection processing of the mobile which 
advances into the above-mentioned monitor field. This penetration detecting 



element the input image acquired in order to the timing from wfiich the 
above-mentioned image sensor differs in time as the 1st image, 2nd image, and ■ 
3rd image — accepting — the difference of the 1st image and the 2nd image — with 
a value the difference of the 2nd image and the 3rd image — the mobile detection 
equipment characterized by calculating^a value, and judging that the mobile 
advanced into the above-mentioned monitor field if the significant data more than a 
constant rate exist in the image data obtained by the AND operation of the 
above-mentioned error-due-to-curvature-and-refraction part value. 
[0005] <Configuration 2> Mobile detection equipment characterized by having the . 
background creation section which updates the background image of a monitor field 
saved for comparison processing each time when a penetration detecting element 
judges that the mobile is not advancing into a monitor field as a result of an AND 
operation in mobile detection equipment given in a configuration 1. 
[0006] <A configuration 3> The mobile detection equipment characterized by.to 
have the template creation section which picks out the image of the lower part of a ' 
mobile from an input image, and creates a template^when the penetration detecting^ 
element which carries out detection processing of the mobile which analyzes the 
image which the image sensor which acquires the image of a fixed monitor field, and 
this image sensor acquired, and advances into the above-mentioned monitor field, 
and this penetration detecting element detect a mobile. 
[0007] <A configuration 4> The mobile detection equipment which carries out 

r ^ ' 

[ having had the template creation section which picks out the image of the specific 
part of a mobile from an input image, and creates a template, and ] as the 
description when the penetration detecting element which carries out detection 
processing of the mobile which analyzes the image which the image sensor which' 
acquires the image of a fixed monitor field; and this image sensor acquired, and 
advances into the above-mentioned monitor field, and this penetration detecting 
element detect a mobile. 

[0008] <A configuration 5> It is mobile detection equipment characterized by to 
create the label image which gave the label value for distinguishing each mobile, 
respectively to two or more mobiles by which the template creation section was 
detected in the image in mobile detection equipment given in a configuration 3, to 
start the part of a label field with the same label value included in this label image 
which includes a lower limit at least, and to create a template. 
[0009] <Configuration 6> It is mobile detection equipment which makes an invalid the 
label value of the label field for the mobile concerned, and is characterized by 



distinguishing from the label field for the mobile which is not yel creating the 
template after the template creation section creates a template in mobile detection 
equipment given in a configuration 5. 

[0010] <Configuration 7> It is mobile detection equipment characterized by for the 
template creation section detecting the image of the lower limit of a mobile from an 
input image in mobile detection equipment given in a configuration 3, and creating 
the template of the mobile when the distance in the image from the lower limit of an 
input image to the lower limit of a mobile is beyond a threshold. 
[001 1] <A configuration 8> It is mobile detection equipment which the template 
creation section detects the image of the lower limit of a mobile from the 
penetration car detection field in an input image in mobile detection equipment given 
in a configuration 3, and is characterized by to cut down the. image of the lower part 
of the mobile from the above-mentioned input image, and to create a template when 
the distance in the image from the lower limit of a penetration car detection field to 
the lower limit of a mobile is beyond a threshold. 

[0012] <Gonfiguration 9> Mobile detection equipment with which the color or 
luminance distribution of the image seen horizontally is characterized by judging the 
part which fulfills predetermined conditions to be the lower part of a mobile in mobile 
detection equipment given iii a configuration 3 when the image of a mobile is 
detected from an input image. 

[0013] <Configuration 10> The image sensor which acquires the image of a fixed 
monitor field with a predetermined frame period, the image of two or more monitor 
fields acquired to the timing from which this image sensor differs in time — that 
difference — it analyzing in quest of a value, and. when the penetration detecting 
element which carries out detection processing of the mobile which advances into 
the above-mentioned monitor field, and this penetration detecting element detect a 
mobile In the image of two or more monitor fields acquired to the timing to which the 
above-mentioned image sensor differs from the template creation section which 
picks out the image of the lower part of a mobile from an input image, and creates a 
template in time Mobile detection equipment characterized by having the rate test 
section which asks for the passing speed of a mobile from the trace section which 
carries out matching processing using the above-mentioned template, and detects 
the movement magnitude of a mobile, and the movement magnitude of a mobile and 
the above-mentioned frame period which the above-mentioned trace section 
detected. 

[0014] <Configuration 11> It is mobile detection equipment characterized by 



receiving an input image with such a short frame period that the trace section 
having many templates in mobile detection equipment given in a configuration 10, 
and receiving an input image with such a long frame period that there being few 
templates. 

[0015] <Configuration 12> It is mobile detection equipment characterized by 
receiving an input image with such a short frame period that the passing speed of 
the mobile by which the trace section was detected in the input image in mobile 
detection equipment given in a configuration 10 being quick, and receiving an input 
image with such a long frame period that passing speed being slow. 
[0016] <Configuration 13> about the image of two or more monitor fields acquired to 
the timing from which the image sensor which acquires the image of a fixed monitor 
field, and this image senisor differ in time that difference — it analyzing in quest of a 
value, and, when the penetration detecting element which carries out detection 
processing of the mobile which advances into the above-mentioned monitor field, 
and this penetration detecting element detect a mobile In the image of two or more 
monitor fields acquired to the timing to which the above-mentioned image sensor 
differs from the template creation section which picks out the image of the lower 
part of a mobile from an input image, and creates a template in time Mobile 
detection equipment characterized by having the rate test section which asks for 
the passing speed of a mobile from the trace section which carries out matching 
processing using the above-mentioned template, and detects the movement 
magnitude of a mobile, and the movement magnitude of a mobile and the 
above-mentioned frame period which the above-mentioned trace section detected. 
[0017] 

[Embodiment of the Invention] Hereafter, the gestalt of operiation of this invention is 
explained using an example. 

<Configuration of the whole equipment> Drawing 1 is the block diagram of the mobile 
detection equipment by this invention. This equipment analyzes the input image 2 
acquired by the image sensor 1 , and performs detection processing of the mobile 
which advances into a monitor field which is explained later. This equipment consists 
of the input image storage section 3, the penetration car detecting element 4, the 
background creation section 5, the initial car template creation section 6, the 
number measurement section 7, the car template storage section 8, the car trace 
section 9 and a rate test section 10, and the transit judging section 1 1. In addition, 
although this invention is applicable to detection processing of not only a car but 
various mobiles, it explains here taking the case of detection processing of the car 
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which runs a path on the street toward one side, for example. 
[0018] First the image sensor 1 is explained. What was introduced by the reference 
already explained in the part of the conventional technique is used for the image 
sensor 1. In addition, the equipment for image acquisition of the various kinds of a 
CCD camera and others can be used. 

[0019] The contents explanatory view of the input image 2 acquired by the image 
sensor 1 by drawing 2 is shown. As shown in (a) of drawing, the image sensor 1 
supervises the monitor fields 14, such as a highway, and photos the mobile of the 
car 15 grade which moves in a it top. The image sensor 1 is installed towards the 
sense to which a car 15 runs from the slanting upper part of the monitor field 14, as 
shown in this drawing. In this way, the image obtained by the image sensor 1 
becomes what **(ed) the posterior part of car 15A which advances the monitor field 
14 top in the direction of arrow-head K, or 15B, as shown in drawing 2 (b). 
[0020] <Outline actuation of equipment> The outline actuation of above mobile 
detection equipment is as follows. The explanatory view of a penetration car 
detection field is shown in drawing 3 . As shown in this drawing, the image sensor 1 
shown in drawing 1 photos the monitor field 14, and obtains an input image as shown 
in this drawing. Here, in order to detect car 15A and 15B which newly advance into 
this monitor field 14 one after another, the penetration car detection field 16 is set 
as the part by the side of the bottom of an input image (i.e., most this side). If 
penetration of car 15A or 15B is detected in this part, the image of the posterior 
part of each car will be cut dowri and a template will be made. And shade pattern 
matching using this template is performed on the input image acquired one after 
another with the passage of time. In this way, a motion of an inter-frame car is 
pursued. 

[0021] It asks for the instantaneous velocity of each car by the elapsed time which 
the migration length of the pursued template and this migration take. Moreover, 
whenever a trace car passes through number measurement Rhine 1 7, the number of 
a car is added up. Furthermore, evasion transit is carried out before trace close 
Rhine 1 8, or a different mobile from the usual operation which carries out 
abnormality transit is detected. 

[0022] The explanatory view of the car detection approach is shown in drawing 4 . 
The one approach of starting the template of the above empty vehicle both [ in an 
input image ] 1 5A and 1 58 is explained here. Here, about an input image, difFerential 
processing which met in the direction of X and the direction of Y is performed, and a 
light-and-darkness pattern is obtained. The graph of drawing shows the result of 




having carried out projection addition of each pixel value in accordance with each 
axis of coordinates, respectively. 

[0023] That is, in the part with many elements of an image, the crest of a graph is 
high. By dividing this with a threshold, the part which shows the projection aggregate 
value beyond a threshold judges it as the location in which a car exists in an input 
image. The part which shows the maximal value of a graph is the core of a car. The 
template of a car can also be started using such a decision result. However, the 
following problems arise by this approach. 

[0024] The explanatory view of a template is shown in drawing 5 . If it sees about 
car 15A of this drawing, as for the template acquired by the above approaches, it is 
desirable that it is an image containing full [ of the car of that number-plate 
circumference ]. It is. because this part is characteristic for every car and it is easy 
to distinguish each. However, depending on the class of car, logging of such a 
template 20 may not be easy. 

[0025] As shown in drawing 4 . when car 15A is oompared with car 15B, car 15A is 
the car of a body color deep as a whole, and car 15B is the car of a comparatively 
bright body color. The light-and~darkness pattern of the car lower part is not always 
strong in such a case. Therefore, when the above-mentioned graph is used, 
depending on the case, the upper part of a car etc. may be started as a template. It 
becomes difficult to perform pattern matching to a template, distinguishing each car 
in such a case, since there are few descriptions. 

[0026] The trouble explanatory view of a large-sized car is shown in drawing 6 . For 
example, like this truck, when it is a car with large die . length and height of a car, 
when the horizontal light-and-darkness pattern of car pars intermedia is weak, pars 
intermedia is missing, and the template of the upper part and the lower part may be 
started separately. That is, counting of the one car may be carried out to two sets. 
A template which such a problem does not produce is started in this invention. 
[0027] <Configuration of each part of equipment> The input image storage section 3 
shown in drawing 1 is constituted by the memory which memorizes above the input 
image 2 which the image sensor 1 acquired continuously to different timing in time 
by three frames. In addition, the input image memorized in this way was used as the 
input images 3A, 3B, and 3C. The 1st image 3A and a degree set to the 2nd image 
3B, and the newest one sets [ the oldest image ] to 3rd image 3C. 
[0028] If the image sensor 1 acquires the image of a monitor field with a 
predetermined frame period, every one frame of the image will be memorized by the 
input image storage section 3. At this time, each time oldest image is driven out and 




it has the composition that a new image is added. 

[0029] The penetration car detecting element 4 accepts 2nd image 3B, three kinds 
of input images, i.e., image 3of ** 1st A, outputted from the input image storage 
section 3, and 3rd image 3C, carries out comparison analysis of these, and has the 
function to judge whether the car newly advanced into the monitor field. In addition, 
if it judges with the car having newly advanced, the penetration car detecting 
element 4 outputs judgment signal 4A, and it is constituted so that creation of a 
template may be required. It is for supervising until the oar passes through a monitor 
field after that The background creation section 5 is a part which performs 
processing which updates the background image of a monitor field which changes 
every moment. 

[0030] That is^ an input image when the penetration car detecting element 4 has not 
detected penetration of a car is acquired, and it considers as a background image. It 
asks for a weighted average with the background image which it is made to 
specifically be shown in the following (1) type, and was accumulated in until, and the 
newly inputted input image. In addition, let the pixel value of the background image of 
the location in an input image (x y) be a background image (x y). 
Background image (x y) = alphaxS 102(1) (x, y)+ betaxS 102(2) (x, y)+ gammaxS 102 
(3)(xy)-(1) 

However, alpha, beta, and gamma are the constants of the arbitration which fills 
"alpha+beta+gamma =1" and alpha>=0. beta,>=0, and gamma>=0. In addition, before 
starting car detection equipment, this background creation section is made to input 
and memorize an early background image beforehand. 

[0031] When it recognizes that the car newly advanced all over the monitor field by 
output 4A of the penetration car detecting element 4, the initial car template 
creation section 6 takes the difference of background-image 5A which the 
background creation section 5 outputs, and input image 38 accepted from the input 
image storage section 3, and has the function to extract the image of the car which 
advanced into the monitor field. Template 6A is created with the image of the lower 
field in the image of this car. whenever [ which outputs one template 6A to 
coincidence ] — signal 6B — the number — counting — it outputs to the isection 7. 
the number — counting — the section 7 has the function in which signal 6B 
performs number measurement of the car which advanced into the monitor field. 
[0032] The car template storage section 8 is a part which was outputted from the 
initial car template creation section 6 and which memorizes template 6A of all 
penetration cars collectively. In addition, the template memorized here is transposed 




to the template updated in the car trace section 9 at any time.'Since the magnitude 
of the image of the car becomes small gradually after the same car advances into a 
monitor field before passing as shown in drawing 2 (b). it is because magnitude of a 
template must also be made small along with it. 

[0033] The car trace section 9 moves the template memorized by the car template 
storage section 8 in an input image, and has the function to detect a motion of a car, 
by carrying out pattern matching. It asks for motion vector 9A of each car by this. 
Moreover, template 9B from which magnitude changed according to the motion of a 
car is newly created, and it has the function outputted to the car template storage 
section 8. 

[0034] This car template 9B is replaced with the template of the already memorized 
car corresponding to the car template storage section 8. And the car trace section 
9 is provided with new template 8A, and pattern matching which used the following 
input image is performed. In addition, since a template becomes unnecessary, the 
car beyond trace close Rhine 18 already explained using drawing 3 is eliminated from 
the car template storage section 8. The car trace section 9 also has the function 
which outputs this erase command. 

[0035] The rate test section 10 accepts motion vector 9A outputted from the car 
trace section 9. and has the function to perform processirijg which measures the 
movement magnitude and passing speed of a car. It has the function to judge 
whether transit of the car is normal in abnormalities based on motion vector 9A to 
which the transit judging section' 11 was outputted from the car trace section 9. 
[0036] Although the above is the outline configuration of equipment, detailed 
configuration and explanation of operation are hereafter given for every functional 
block of equipment. 

<Penetration car detecting element> The functional block diagram of a penetration 
car detecting element is shown in drawing 7 . The penetration car detecting element 
4 has the function to accept and process the 1st image 3A acquired from the input 
image storage section 3 to timing [ time little by little ] different, respectively, the 
2nd image 3B, and 3rd image 3C, as shown in drawing. For this actuation, difference 
Wakebe 21 and 22, the binary-ized sections 23 and 24, the AND-operation section 
25, and the judgment section 26 are formed. 

[0037] in addition, difference — the section 21 ~ the difference of the 1st image 
and the 2nd image — it is the part which performs data processing which calculates 
a value, difference — the section 22 ~ the difference of the 2nd image and the 3rd 
image — it is the part which performs data processing which calculates a value. 




Difference Wakebe's 21 output 21 A is binary — ization-processed in the binary-ized 
section 23. Difference Wakebe's 22 output 22A is binary — ization-processed in the 
binary-ized section 24. And in the AND-operation section 25. AND processing of the 
outputs 23A and 24A of the binary-ized sections 23 and 24 is carried out, and 
output 25A is obtained. 

[0038] the above — difference — processing is performed in order to extract the 
part which changes in time in an input image, difference — the difference per pixel 
of the location where each input image is the same in processing — a value may be 
calculated, and an input image may be divided into the block of for example, 2x2 
grade, the pixel value of a block of the same location may be totaled, and you may 
ask for the difference of the average. Binary-ized processing is changed into the 
image with which each pixel value is expressed with binary [ of 0 and 1 ] using a 
predetermined threshold in difference Wakebe 21 or the output of 22. It becomes 
the operation "1". if each pixel value is under a threshold and it is "0" and beyond a 
threshold. 

[0039] The explanatory view of a penetration car detecting element of operation is 
shown in drawing 8 . It will be set to AND image output 25A as shown in the bottom 
of drawing, if an AND with output 24A which took and made binary 1st image 3A, 
image 23A which made difference of 2nd image 3B binary, 2hd image 38, and 
difference of 3rd image 3C is taken as shown in this drawing. Thereby, the result 
with some which move in the inside of a background image that it can Judge is 
obtained from any two outputs of difference Wakebe 21 and 22. That is, a pixel value 
can judge that a mobile exists in an input image if the field of "1" exists more than 
fixed in this image. 

[0040] If 1st image 3A and 2nd image 38 which are an input image^ and 3rd image 3C 
are completely only changeless background images, respectively, as for both the 
outputs 23A and 24A that made the difference binary, all pixel values are set to "0." 
If the AND is taken, as for all outputs, a pixel value is set to "0", and it can be 
judged that a mobile does not exist in an input image, if the mobile of the area more 
than fixed advances into a monitor field — each input image — respectively — 
difference — if a value arises and the AND is searched for, the pixel value more 
than fixed will arise [ the field of ], and this will be detected. 
[0041] If penetration of a car is judged by such approach, even if a background will 
change due to sunshine in time, or even if it changes with change of the weather 
gradually, penetration of a mobile can be detected without the being influenced. It is 
because the time amount for acquiring 1 st image 3A, 2nd image 38, and 3rd image 




3C is such time amount short enough that a time change of a background can be 
disregarded. Moreover, by setting up a suitable threshold, there is also little 
incorrect detection of a noise and it ends. In addition, the background image is 
updated when a car does not exist all over a monitor field. 

[0042] Moreover, the time of a car not existing all over a monitor field is detected, a 
background image is acquired, and if it is made to perform a weighted average with 
the background image memorized immediately before in the already explained way, 
the contents fluctuation of a background image and the effect of a noise by various 
factors are fully mitigable. 

[0043] The <initial car template creation section>: Drawing 9 is the block diagram of 
the initial car template creation section, the initial car template creation section 6 
consists of difference Wakebe 31, the binary-ized section 32, the label image 
creation section 33, the template logging section 34, and the label image storage 
section 35. Input image 3B and background-image 5A which are shown in drawing 1 
input into this initial car template creation section 6. And bnly when input image 38 - 
inputs, this part operates. 

[0044] difference — the section 31 ^- the difference of input image 3B and 
background-image 5A — it is the part which carries out data processing of the 
value and obtains output 31 A. difference — a value is calculated for excepting the 
part equivalent to a background image from an input image, and asking for the image 
of a car part, this difference — the difference per pixel of the location where input 
image 3B and backgroundHmage 5A of processing are the same — the block which 
could calculate the value and consisted of 8x8 pixels in input image 38 and 
background-image 5A — dividing — the difference of the average of a block of the 
same location — a value may be calculated. 

[0045] Moreover, each differential image of input image 38 and background-image 
5A may be made, and difference of differential images may be performed, after 
performing orthogonal transformation for dividing into a BxB-pixei block, for example, 
respectively, and obtaining each characteristic quantity — the difference of those 
block characteristic quantity — a value may be calculated. The approach shown, for 
example in the following reference as an approach of calculating this block 
characteristic quantity is adopted. "Ago-in **, processing of an image and Nagao 
Tomoharu "recognition" Shokodo (P. 47-P.58)" 

[0046] The binary-ized section 32 makes binary image 31 A outputted by difference 
Wakebe 31 with a threshold, and outputs binary-ized image 32A; The label image 
creation section 33 is a part which performs labeling which the pixel value of 



binary-ized image 32A assigns an independent different number for every set about 
the part of "1." The part of "1" is a part in which the image of a car exists [ a pixel 
value ]. In this way, label image 33A is generated. This concrete art is indicated by 
the following reference. "Ago-in **, processing of an image and Nagao Tomoharu 
"recognition" Shokodo (P. 67-P.68r 

[0047] The label image creation section 33 creates the label image which has the 
following pixel values using such labeling technique. First, different numbers, such as 
"0", "l", and "2", are assigned to the field to which a car exists for every 
independent set. It is for attaching and distinguishing a different label on a different 
car. In this way, the total of the assigned field will be called label several n. Moreover, 

" is assigned to the part in which a car does not exist as a label value. Drawing 
10 explains the result to a detail later. 

[0048] The template logging section 34 accepts input image 3B and label image 33A 
which the label image creation section 33 outputs, receives the label image further 
memorized before the label image storage section 35 memorized, and investigates 
the contents of each label field of a label image. And it judges whether the car lower 
part is contained to the label field here. A template will be started if the car lower 
part is contained. Moreover, the signal for a car count whenever it starts a template 
is generated. In this way, template 6A and passage car signal 6B are outputted from 
the template logging section 34; 

[0049] The explanatory view of a label image' is shown in drawing 1 0 . As for label 
image 33A shown in drawing, the' whole is beforehand initialized by the label value 

This is created about the part of the penetration car detection field in an input 
image. If a car is detected by the part which shows hatching of drawing, the label 
value of the label field on the left-hand side of drawing will be set to "0", for 
example, and the label value of a right-hand side label field will be set to "M' And 
the condition of the lower limit is investigated about each label field. 
[0050] If the lower limit of each label field is among an image, for example, is distant 
from the lower limit of label image 33A one or more [ L ], it will be judged that the 
image of the car lower part is contained in label image 33 A. Based on this decision, 
the image of the car specified by this label field is started from input image 3B. In 
this way, the template containing the car lower part is created. 
[0051] After starting a template, the label field of the label vdue "0" of a label image 
is switched to the contents, finishing [ template logging ]. That is, a label value is 
changed into "-2" from "0." Therefore, the processing which judges whether the 
lower limit of the label field exists in the place where the label value looked for a 




part for "1" or the other label value part, and separated only Lf from the lower limit 
of label image 33A after that is repeated. 

[0052] In this way, rewritten label image 33A turns into output 34A of the template 
logging section 34, and is again memorized by the label image storage section 35. 
And it is used for the continuing processing. 

[0053] Although the above is outline actuation of the initial car template creation 
section, detailed actuation is explained using the following flow chart. Drawing 1 1 is 
the operation flow chart of the initial car template creation section. First, in step SI, 
a new label image inputs into the template logging section 34 from the label image 
creation section 33. The number of labels is set to n. Parameter i is initialized to 
"0." This parameter i is a counter forjudging whether processing was ended about 
all label images. 

[0054] In the following step S3, it judges whether i is smaller than n, and if small; it 
will progress to step S4. In step S4, it investigates with which label field of a label 
image just before already being stored in the label image storage section 35 it 
corresponds about the field whose label value of a label image is i. In the label field 
in which the label field of i corresponds [ the label value in a new label image ] in the 
last label image, if a label value is the thing of "-2", it will progress to step SI 2. 
Since this is already creation settled about a template, it is for passing processing. 
Here, the pixel value of a label field (i) is changed into "-2." In addition, as an 
approach ofinvestigating a corresponding label field, when a label field (i) is laid on 
top of the last label image, an ovierlapping area will be the label field where it 
corresponds, if the label field beyond a threshold exists in the last label image. 
[0055] When other, it progresses to step S5, and. it judges whether the label field of 
the label value i has separated L 1 pixels or more from the lower limit of a 
penetration car detection field. If separated, it will judge that the lower limit of a car 
has appeared all over a penetration car detection field^and will progress to step S6. 
Here, the label value of an input image starts the field applicable to i, and creates a 
template. And it progresses to step S7 and a template is evaluated. This evaluation 
is the thing of the contents whether it is specifically bilateral symmetry or the tail 
lamp is included 

[0056] in addition, the part to which the approach of starting corresponds to a label 
field — the rectangle as which a form as it is is sufficient and which is carried out 
and is circumscribed to a label field is sufficient. Moreover, you may carry out as a 
part for fixed size is started from the location of the lower limit of a label field in an 
input image. 




[0057] The evaluation actuation explanatory view of the car lower part is shown in 
drawing 1 2 . As shown in drawing, when a template 20 is extracted from the lower 
part of car 15A, symmetric property on either side is investigated first, this — the 
difference of the pixel value of the core of the axis of abscissa (horizontal shaft) of 
a template to the right-and-left equal distance — a value can be accumulated and 
value B/a which broke the accumulation value P by the area a of a template can 
carry out by decision of being smaller than a fixed threshold, if it is bilateral 
symmetry — difference — it is because a value is set to "0." 
[0058] Moreover, a tail lamp exists in the car lower part. Therefore, distinction of 
tail-lamp existence leads to evaluation of whether.to. have extracted the template of 
the car lower part, in this case, a ******** [ that the dark part beyond a threshold 
is in both ends using color distribution of a lateral (horizontal) red component as 
shown in drawing ] — ** — the judgment to say is performed. If an image is a 
monochrome shade image, daytime's tail lamp will be black and will be reflected, 
brightly night. If this is used, it can judge whether it is the car lower part according 
to lateral shade distribution. 

[0059] The related explanatory view of a label field and a template is shown in 
drawing 13 , To drawing, the case where a template is started from corresponding 
input image 3B is considered about a certain label field in the label image 36 so that 
it may be shown. It is made to correspond to the penetration car detection field 16 
of drawing, and label image 33A is generated. Suppose that the car lower part was 
detected in label image 33A in this conditiop. In this case, the field for height of h of 
the upper part [ y ] is cut out from input image 38 from the location equivalent to a 
label image lower limit, and a template 20 is created. What is necessary is just to 
determine this h experientially in quest of the suitable value which can cut down the 
image containing the car lower part. 

[0060] It returns to the flow chart of drawing 1 1 again, and. a template judges 
******** [ as the car lower part ] that step S8 already explained. In step S9. if it is 
judged that it is as the car lower part, template 6A and passage car signal 68 will be 
outputted. Template 6A is outputted towards the car template storage section 8. 
shown in drawing 1 . Moreover, passage car signal 68 is outputted towards the 
number measurement section 7. 

[0061] Jn the following step S10, a label value changes the pixel value of the field of i 
into "-2." In this way, after indicating that it is template extract ending, return and i. 
are incremented to step S11. After that, processing about the following image is 
again performed from step S3. After the processing about all images is completed, it 




progresses to step SI 3 and the label image after processing is memorized in the 
label image storage section 35. 

[0062] The <car trace section> Next, actuation of the car trace section is explained. 
The car trace section is a part which performs processing which uses the template 
corresponding to each car memorized by the car template storage section 8 shown 
in drawing 1 , detects the location of the car in an input image, and detects the 
passing speed etc. 

[0063] The explanatory view of template matching which shows the principle of 
operation of the car trace section 9 to drawing 14 is shown. As shown in drawing, it 
detects using the template 20 which shows that car 15A moves to the left-hand 
side of drawing in input image 3B. Input image 3B is updated for every fixed time 
amount The newly inputted input image is the thing of the contents which car 15A 
moved. About each input image, the location of the migration place of a car is 
detected using a template 20. As shown in drawing, it matches with this by moving a 
template 20 all around. 

[0064] as concrete processing — the identification number of a template (i), i.e., a 
car, ^ — the field of i — the accumulation between input images — difference — a 
value is computed, and accumulation — difference — movement magnitude in case 
a value is min is outputted. The location of the car with which it corresponds in an 
image is detected by this. If this location is detected for every input image and the 
movement magnitude of a template is investigated each time, the passing speed and 
the migration condition of a car are detectable. 

[0065] Next, actuation of the car trace section is explained using a flow chart 
Drawing 1 5 is the operation flow chart of the car trace section. First, processing will 
be started, if new input image 3A inputs into the car trace section 9 shown in 
drawing 1 and the template about each car in an input image is memorized by the 
car template storage section 8. 

[0066] In the introduction step SI, i is reset to "0." This i is a parameter for 
identifying a template. That is. T templates shall exist here. At step S2, it judges 
whether i exceeded this T and, in the case of not more than it progresses at step 
S3. It is decision whether processing was ended about all templates. And it judges 
whether the template applicable to an identification number i exists at step S3. If it 
does not exist it jumps to step S10, and i is incremented, and another template is 
looked for. 

[0067] At the following step S4, matching processing of a template is performed in 
the already explained way. At step S5, it judges whether a matching score is better 




than a threshold RO. the difference explained by drawing 1 4 — the value which 
broke the accumulation value by area of a template is called a matching score. It 
judges whether this matching score is over the threshold. With the threshold 
[ below ], template detection conditions are fulfilled. It is shown that the agreement 
nature of an input image and a template is so good that a matching score is small, 
and the dependability of a matching vector is high. 

[0068] If a matching vector is outputted at step S5, it will progress to step S7 and 
will judge whether the car crossed trace close Rhine. This processing is repeated 
until the detection location of a car crosses trace close Rhine 1 8 shown in drawing 
3 . Then, it judges whether the template (i) fulfills updating conditions. According to 
migration of a car, the image of the same car becomes small gradually. According to 
this, it adjusts so that a template may also contract size. This is renewal of a 
template. 

[0069] The updating conditions of a template judge whether a matching score is 
larger than the somewhat larger threshold R1 than the threshold used at step S5, 
and are decided. It uses that a matching score gets somewhat bad with migration of 
a car. Moreover, it judges whether the size of the vertical direction of a matching 
vector is larger than a threshold LV. The movement magnitude of the vertical 
direction and the reduction percentage of a template are biecause it has a fixed 
relation. Since the form of a template will change if any conditions are fulfilled, a 
template is updated (step S9). When there 1$ no need of updating, it returns to step 
SI 0 and moves to other template-matching processings. 
[0070] In a template update process of step S9, an old template is moved by the 
matching vector and it piles up on an input image. And the frame size of a template 
is reduced according to the movement magnitude, and the image of the size is cut 
down from an input image. Let the cut-down image be a new template. It memorizes 
in the car template storage section 8 which showed this template and its location to 
drawing 1 , and an old template is eliminated. In addition, at step S7, when it is 
judged that trace close Rhine 18 was crossed, the corresponding template eliminates 
this in step S1 1 for used. 

[0071] The explanatory view of the template reducing method is shown in drawing 
16 . About input image 3A shown in drawing, as shown in drawing, the Z-axis is set 
as a lengthwise direction, and the location on this shaft is shown in drawing — as — 
hO and h — 1,0, and hv — as — it sets up. Let the part of 0 be the zero of the 
Z-axis. Here, the location of the template after PO and migration is set to PI for the 
location of the template before migration. Thus, if a car moves toward the location 




of hO to hi toward the back from this side, the image of a car will become small due 
to perspective. 

[0072] Therefore, it is necessary to change size by the thing before the template for 
detecting the location of a car also moving, and the thing after migration. Then, a 
template is reduced. This reduction percentage comes to be shown in the following 
formula (2). 

R(hO, hi) =(h1-hv)/(hO-"hv) — (2) 

R (hO, hi) is template reduction percentage when a car moves to the location of hO 
to hi. (hl-hv) is the die length from hi to hv, and (hO-hv) is the die length from hO 
to hv. In a trace of ^ car, such processing is also taken into consideration and a 
movement vector is detected. 

[0073] <Rate test section> If the above movement vectors are detected, the 
passing speed of a mobile, i.e., the passing speed of a car, can be shortly measured 
with a distance in the meantime and the relation of time amount. 
[0074] The explanatory view of a rate test section of operation is shown in drawing 
1_7 . As already explained using drawing 2 , the image sensor 1 is photoed so that the 
monitor field 14 may be aslant looked down on from the location of height H, as 
shown, for example in this drawing. The rate of a car is found from a matching 
vector including the positional information of each car in the input image 3 shown in 
(b) of drawing. The rate of a certain car is set to S. In this case, if the location of 
the car in the input image acquired to different time amount t1 and t2 is set to L 
(t1) and L (t2). the rate s of a car can be found by the following (4) formulas. 
S=(L(t2)-L(t1))/(t2-t1)-(3) 

[0075] From the location of the car in the input image 3, it is performed as follows in 
quest of the location of a car path on the street. Now, the installation location of a 
camera and the relation of the monitor field 14 (this is equivalent to the road 
surface of a road) presuppose that it is in a condition as shown in drawing 1 7 (a). 
Suppose that an inclination in case a car is in the location of Rm is thetam. The 
actual location of central Rhine Lm in input image 3A shown in drawing 1 7 (b) is a 
distant location Rm horizontally from the image sensor 1 of drawing 1 7 (a). 
[0076] Moreover, the location of a car when penetration is first detected in the input 
screen of (b) is Lo.rThe distance between the image sensor 1 and Rhine Lm is OM 
on this input screen. The location of Rhine Lo shown in (b) path on the street is a 
distant location Ro horizontally from directly under [ of the image sensor 1 shown in 
(a) ]. The angle of the line which connects the image sensor 1 and this Ro, and a 
horizontal line to make is set to thetao as shown in drawing. 




[0077] At this time, it can ask for the location Rx of the car in the place distant 
from Rhine Ro of input image 3A shown in (b) Lx path on the street by. the following 
(4) types. 

Rx=H-i:an [pi / 2-thetam-tan -1 {A-tan(thetao-thetam)/OM-Lxl] 

- (4) 

However, they are thetam=tan -1 (H/Rm) and thetao=tan -1 (H/Ro). 

[0078] The explanatory view (the 2) of a rate test section of operation is shown in 

drawing 1 8 . (a) of this drawing — the location of the mobile of the monitor field 1 4 

— Ra (i) — as — (i= 0, 1, 2 and 3, — ) it is expressing. This location was displayed 
like a (i). i= 0. 1 and 2, and — into input image 3 A of (b) of drawing. Thus, when 
asking for relation with the location of an actual location and an input image top 
beforehand and a car is detected in the location X of arbitration, an actual location 
can be correctly calculated using interpolation algorithms, such as spline 
interpolation known well. 

[0079] Thus, an actual location is calculated, the migration length of a mobile, i.e., a 
car, is calculated, and the rate of a mobile will be found if it converts by the time 
amount carried out based on the switch period (frame period) of the time amount, 
i.e., an input image, which passed in the meantime. 

[0080] The <transit judging section> As mentioned above, if the rate of a car etc. is 
calculated, the run state of a car is analyzable by actually investigating the 
movement vector of the car. The explanatory view of the transit judging section of 
operation is shown in drawing 19 '. It may be going on to the case where for example, 
car 15A is advancing in the arrow-head K1 direction and arrow-head K 2-way of this 
drawing. When moving in the arrow-head K1 direction on the same lane and a lane is 
changed to normal and an arrow-head K 2-way, it judges that this is unusual. In 
various cases, such a result can be used. 

[0081] <Modif!cation> The modification block diagram of the mobile detection 
equipment by this invention is shown in drawing 20 . The equipment of this drawing 
adds the frame periodic-control section 12 to the equipment shown in drawing 1 . 
Other parts are the same configurations and attached the same sign. 
[0082] This frame periodic-control section 12 accepts the rate information on the 
number of car templates memorized by car template storage section 8, i.e., the 
number of the car which exists between trace close Rhine from a penetration car 
detection field, and each car outputted from the rate test section 10, and operates. 
And the frame period corresponding to the rate of a car or the number of a car is 
determined, and a control signal is outputted to the input image storage section 3 or 




the rate setting section 1 0. 

[0083] For example, when the rate of a car is very quick, a car is pursued at short 
spacing, and it must ask for a movement vector, switching an input screen a short 
period. On the other hand, when the rate of a car is slow enough, even if it switches 
a frame a quick period, useless data processing increases and a load increases. It is 
desirable to lengthen a frame period and to process efficiently. 
[0084] Moreover, since the data-processing time amount for processing one frame 
becomes long when there are many cars which exist all over the field which should 
detect a car, i.e., the number of car templates, unless it sets up a frame period a 
little long, data processing does not meet the deadline. In consideration of such a 
case, efficient car detection is attained by attaining optimization of a frame period. 
[0085] The frame periodic-control section 1 2 changes a frame period into twice as 
many 2C as this, when the number for example, of car templates is "0" and the 
standard frame period set up beforehand is set to C. Moreover, if a judgment below 
of a threshold with the maximum velocity of all the cars currently pursued is made, a 
frame period will be similarly set to 2C. On the other hand, a frame period is made 
into C/2 when it becomes beyond a threshold with the number of car templates. 
[0086] The input image storage section 3 has received the input image 2 by fixed 
frame period 0 from the image sensor 1. For example, in lengthening a frame period, 
it controls to choose an input image once and to memorize to 2 times. When 
increasing a frame period 3 times, an input image is memorized once to 3 times, and 
other input images are discarded. Thus, a frame period can be controlled and it can 
process at a suitable rate. 

[0087] In addition, although various thresholds for actuation of each 
above-mentioned functional block were set up. according to the sunshine condition 
of a monitor field, weather conditions, and other conditions, these can be changed 
freely and can be optimized. Moreover, for example in a part which has little 
fluctuation of background images, such as a tunnel, control which is fixed may be 
performed without updating a background image. Furthermore, it is also possible to 
make a penetration car detection field into plurality. 

[0088] The explanatory view of two or more penetration car detection fields is 
shown in drawing 21 . As shown in this drawing, one penetration car detection field 
was set as the input image 3 until now. However, as shown in drawing here, two 
penetration car detection fields 40 and 41 are set up. Thus, a separate penetration 
car detection field is prepared for every lane, and even if it is made to perform 
processing explained until now, it does not interfere. Furthermore, by the 




above-mentioned explanation, the car which runs a path on the street as an 
example of a mobile was mentioned, and it was explained that the template 
containing the image of the car lower part was obtained. However, according to the 
class of mobile, without being limited to the lower part, since it differs, for every 
mobile, the description part specifies the particular part, and can create a template. 
[0089] 

[Effect of the Invention] as mentioned above, the case where the template of the 
mobile which should be pursued from an input image is created — difference with a 
background — a car will not be divided into plurality if an object is specified in quest 
of a value Furthermore, incorrect detection can be prevented if the template 
centering on the car lower part is started being conscious of the description of the 
car lower part Moreover, if a penetration car detection field is set up under the 
input screen as mentioned above, before the cars of order overlap in a screen, 
penetration of a car and logging of a template can be performed. 
[0090] furthermore, the difference of the input screen which gets decision whether 
the car advanced into the screen mixed up in time — if it carries out by carrying out 
based on a value, it is not influenced by various fluctuation of a background image, 
but the migration judging of a car can be performed correctly. 
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[Brief Description of the Drawings] 

[Drawing 1] It is the block diagram of the mobile detection equipment by this 
invention. 

[Drawing 2] It is the contents explanatory view of an input image. 

[Drawing 3] It is the explanatory view of a penetration car detection field. 

[Drawing 4] It is the explanatory view of the car detection approach. 

[Drawing 5] It is the explanatory view of a template. 

[Drawing 6] It is the trouble explanatory view of a large-sized car. 

[Drawing 7] It is the block diagram of a penetration car detecting element. 

[Drawing 8] It is the explanatory view of a penetration car detecting element of 

operation. 

[Drawing 9] It is the block diagram of the initial car template creation section. 
[Drawing 10] It is the explanatory view of a label image. 

[Drawing 11] It is the operation flow chart of the initial car template creation section. 




[Drawing 12] It is the evaluation actuation explanatory view of the car lower part 
[Drawing 13] It is the explanatory view of the template logging approach. 
[Drawing 14] It is the explanatory view of template matching. 
[Drawing 1 5] It is the operation flow chart of the car trace section. 
[Drawing 1 6] It is the explanatory view of the template reducing method. 
[Drawing 1 7] It is the explanatory view (the 1) of a rate test section of operation. 
[Drawing 18] It is the explanatory view (the 2) of a rate test section of operation. 
[Drawing 1 9] It is the explanatory view of the transit judging section of operation. 
[Drawing 20] It is the modification block diagram of the mobile detection equipment 
by this invention. 

[Drawing 21] It is the explanatory view of two penetration car detection fields. 
[Description of Notations] 

1 Image Sensor 

2 Input Image 

3 Input Image Storage Section 

4 Penetration Car Detecting Element (Penetration Detecting Element) 

5 Background Creation Section 

6 Initial Car Template Creation Section (Template Creation Section) 

7 Number Measurement Section 

8 Car Template Storage Section (Template Storage Section) 

9 Car Trace Section (Trace Section) 

10 Rate Test Section 

1 1 Transit Judging Section 



